Lecture 9 : Linear Bandits ( Part II )

There exist an elliptical confidence region for the w, as described in the following theorem Theorem 1. ([2], Theorem 2) Assuming ‖w‖ ≤ √ d and ‖xt‖ ≤ √ d, with probably 1− δ, we have w ∈ Ct, where Ct = { z : ‖z − ŵt‖Mt ≤ 2 √ d log Td δ } For any x ∈ A, we define UCBx,t = maxz∈Ct z′x if w ∈ Ct (which holds with high probability). At each time, the UCB algorithm then simply picks the bandit with the highest UCB given all previous observation. xt = arg max x∈A UCBx,t−1 = arg max x∈A,z∈Ct−1 x′z